• MATB24

    Prof. Michael Cavers

    W1 (3.1)

    Def. Vector Space

    Note: vectors could refer to matrices, polynomials, functions, power series, etc.

    Q Let V={vRn|components of v sum to 0} with usual operations. Is V a vector space?

    A Let u,vV. Suppose u=[u1un],v=[v1vn]

    Check closure under addition:

    u+v=i=1n(ui+vi)=i=1nui+i=1nvi=0+0=0V

    Check closure under scalar mult:

    ru=i=1n(rui)=ri=1nui=r0=0V

    Usual ops A1-A4, S1-S4 all hold V is a vector space.

    Q Find the zero vector in V=R2 where addition is defined by [ab]+[cd]=[a+cb+d+1]

    A [xy]+[ab]=[ab][x+ay+b+1]=[ab]x=0,y=10=[01]

    To define a vector space, need to specify 4 parts:

    1. Objects that make up the set V
    2. The field where scalars come from (if the field of scalars is the set of real #s, then V is a real vector space)
    3. How to add vectors
    4. How to multiply scalars with objects in V

    The below sets with given rules for , are vector spaces.

    NotationDefinition
    Rnset of vectors of length n with real entries
    Cnset of vectors of length n with complex entries
    Pset of all polynomials in x with real coefficients
    PnP with degree n
    Mmnset of all real matrices of size m(rows) by n(columns)
    Fset of real-valued functions with domain R

    Thm 3.1 (Elementary Properties of Vector Spaces)

    Every vector space V has the following properties:

    1. 0 is unique for all vectors in V
    2. v is unique for each vector in V
    3. u+v=u+wv=w
    4. 0v=0,  vV
    5. r0=0,  rR
    6. (r)v=r(v)=(rv),  rR,vV

    Proof

    1. Suppose 2 zero vectors, x and y. Then for all v in V, x+v=v and y+v=v.

      Since yV, sub v=y into x+v=vx+y=y

      Since xV, sub v=x into y+v=vy+x=x

      x+y=y+xy=x0 is unique

    2. By definition, v+(v)=0. Suppose v+u=0.

      u=u+0=u+[v+(v)]=(u+v)+(v)=0+(v)=(v)

      Any u=v additive inverse is unique

    3. If u+v = u+w

      Then v=0+v=(u+u)+v=u+(u+v) = u+(u+w)=(u+u)+w=0+w=w

      Equate LHS & RHS v=w

    4. 0+0v=0v=(0+0)v=0v+0v.

      Equate LHS & RHS 0=0v

    5. rv+r0=r(v+0)=rv=rv+0.

      Equate LHS & RHS r0=0

    6. rv+(rv)=(r+(r))v=0v=0(rv)=(r)v

      v+(v)=0rv+r(v)=0(rv)=r(v)

      (rv)=(r)v=r(v)

    W2 (3.2)

    Vector Space vs Real Space

    E.g. The vector space P of all polynomials is spanned by the set of monomials M={1,x,x2,...,xn,...}

    • M is a basis for P M spans P P = span(M)
    • M generates P and so P is NOT finitely generated

    Def. Linear Combination

    x is a linear combination of x1,x2,...,xkV if there scalars c1,c2,...ckR s.t. x=c1x1+...+ckxk

    Note linear combinations must be of a finite number of vectors (otherwise called Schauder basis)

    Def. Span

    sp(X) is the set of all linear combinations of elements in X, where X is a subset of V.

    If V=sp(X), then the vectors in X span or generate V.

    Q Is 1+x2sp(1+x,x+x2,2+x2)? Assume standard polynomial addition and scalar mult.

    A Check if 1+x2 can be written as a(1+x)+b(x+x2)+c(2+x2)

    1+x2=a+ax+bx+bx2+2c+cx2

    {1=a+2c0=a+b1=b+c (derived using the coefficients)

    {a=1/3b=1/3c=2/3 Unique solution, so yes, 1+x2 is in the span of the set.

    Thm (Span of a subset is a subspace)

    If X is a non-empty subset of vector space V, then sp(X) is a subspace of V.

    Proof

    1. Since X is non-empty, sp(X) must be non-empty

    2. Let r1,r2R, let v1,v2sp(X)

      Since v1,v2sp(X), a1,...,anR,b1,...bnR,x1,...,xnX s.t.

      v1=a1x1++anxn

      v2=b1x1++bnxn

      Then r1v1+r2v2=r1(a1x1++anxn)+r2(b1x1+...+bnxn)

      =(r1a1+r2b1)x1+...+(r1an+r2bn)xnsp(X)

    Non-empty, closed under additional and scalar mult -> span of X is a subspace of V.

    E.g. Let X={1,x,x2}P

    (1+x)+(1+x2)=x2+x+2, which is still a linear combination of vectors in X, so we have closure under addition.

    r(1+x)=r+rx, which is a multiple of - again - linear combinations of vectors in X, so we have closure under mult.

    Corollary (Not covered in class)

    sp(X) is the smallest subspace of V containing all the vectors in X.

    Proof

    Let M be the smallest subspace of V containing v1,...,vn

    Claim: M=span(v1,,vn)

    Since vi=0v1+...+1vi+...+0vn for all 1in,

    we have vispan(v1,...,vn) by definition of span, so Mspan(v1,...vn)

    Since viM and every subspace is a vector space with closure properties,

    we have a1v1+...+anvnM, so span(v1,vn)M

    Thus M=span(v1,,vn)

    Def. Subspace

    Let V be a vector space and U a subset of V. Then U is a subspace of V if U is a vector space using the defined ops for V.

    E.g. Counter examples

    Subsets of R2 that are closed under addition but not scalar multiplication:

    S={[xy]:x,y0} if multiply by r=-1, the result 0 soS

    S={(n,0)R2:nZ} if multiply by r = 0.5, the result Z

    S={(r,s)R2:r,sQ} if multiply by r = π, the result Q

     

    Subset of R2 which is closed under scalar multiplication but not addition:

    {[xy]:x=0 or y=0}

    Ways to prove U is a subspace

    1. Show U is a vector space using the definition (too much work)
    2. Use the span theorem (probably easiest)
    3. Use the subspace test

    Thm 3.2 (The Subspace Test)

    A subset U of V is a subspace of V if

    1. The zero vector 0U (Sometimes omitted as it is implied by 3)
    2. U is closed under vector addition
    3. U is closed under scalar multiplication

    Shortcut for 2 & 3: check both at the same time by using ax+byU

    Q Is the set of nxn invertible matrices a subspace of Mn (the vector space of all nxn matrices)?

    A No. The sum of 2 invertible matrices may not be invertible (violates 2).

    Also, the 0 matrix is not invertible (violates 1).

    Def. Linear (In)dependence

    Let X be a set of vectors in a vector space V. (X can be an infinite set.)

    A dependence relation in X is an equation of the form r1v1+...+rkvk=0 with some ri0 where viX for i = 1, … , k.

    If such a dependence relation exists, then X is linearly dependent. Otherwise, X is linearly independent.

    If X is finite, then the following definition applies:

    The set {x1,...,xk} with x1,...,xkV is linearly independent if c1x1+...+ckxk=0 implies c1=...=ck=0

    Note: { } is independent (vacuous statement)

    • There is no collection of vectors from { } satisfying a dependence relation because there is nothing in the set at all
    • sp({ }) = {x:x is the linear combination of vectors in { }an empty sum}
    • Convention: the empty summation is taken to be its additive identity (0).

    Q Suppose {x,y,z} is linearly independent. Prove {x+y,2x+z,y5z} is also linearly independent.

    A Assume c1(x+y)+c2(2x+z)+c3(y5z)=0

    WTS this implies c1=c2=c3=0

    c1x+c1y+2c2x+c2z+c3y5c3z=0

    (c1+c2)x+(c1+c3)y+(c25c3)z=0

    Since {x,y,z} is linearly independent, {c1+2c2=0c1+c3=0c25c3=0c1,c2,c3=0independent

    Q Consider F, the vector space of all functions from RR (with usual operations).

    Is X={sin2x,cos2x,1} linearly independent?

    A X is linearly dependent since (1)sin2x+(1)cos2x+(1)(1)=0

    Q Is X={sinx,cosx} linearly independent?

    A X is linearly independent.

    Assume asinx+bcosx=0 holds x

    (The equation must hold for x=0,π/2 since it holds for all x.)

    asin0+bcos0=0b=0

    asin(π/2)+bcos(π/2)=0a=0

    a,b=0 X is linearly independent

    If we find a non-trivial solution (a or b or both != 0), NO conclusion can be made.

    To generate more equations, we can take derivatives first then plug in points.

    To check if a set is linearly independent:

    If it cannot be observed, set up the dependence relation equation as the first step.

    If the set spans a space with a dimension lower than the cardinality of the set (i.e. dim(span of the set) < |set|), then the set cannot be linearly independent.

    If given a set of polynomials, put the coefficients as column vectors in a matrix, reduce to REF, and solve the system of equations.

    Other ways:

    Def. Basis

    A subset X of V is a basis for V if:

    1. X spans V
    2. X is linearly independent

    Def. Dimension

    Let V be a finitely generated vector space. The number of elements in a basis for V is the dimension of V, dim(V).

    Note: When giving definitions, "a" and "the" imply different things - "the" implies that the object described is unique.

    Q Is dimension well defined? I.e. Can 2 bases have different sizes?

    A It is well defined since any 2 bases of V must have the same number of elements. This is true by the following theorem.

    Thm (Dimension and Linear Independence)

    Suppose that V is a vector space of dimension n. Then any subset X with size greater than n is linearly dependent.

    Proof

    Since V has dimension n, there must exist a basis of size n: B={v1,...,vn}

    Let X={x1,...,xm} with m>n

    Since B is a basis, each xj can be written as a linear combination of elements of B: xi=j=1najivj,i{1,,m}.

    i.e. x1=a11v1+...+an1vn

    xm=a1mv1+...+anmvn

    Assume a vector (λ1,...,λm) such that [a11...a1man1...anm][λ1λm]=[00]

    Since m > n, we have more unknowns than equations, so there must be a non-trivial (non-zero) solution, i.e. the lambda vector must be non-zero.

    Setup dependence relation: i=1mλixi=0

    i=1mλixi=i=1mλi(j=1najivj)=j=1n(i=1mλiaji)vj=optionalj=1n(λ1aj1+...+λmajm)vj=j=1nλ1aj1vj+...+λmajmvj

    aji cannot all be 0, and λi cannot all be 0, so there exists at least one non-zero coefficient in the dependence relation
    so X is not linearly independent.

    E.g. Rn (with usual operations) has a standard basis of {[1000],[0100],...,[0000]} and dim(Rn)=n

    E.g. Mm,n (with usual operations) has dim(Mm,n)=mn

    E.g. Diagonal matrices Mn (with usual operations) has dim(Mn)=n

    E.g. Pn (with usual operations) has a standard basis of {1,x,...,xn} of dim(Pn)=n+1

    Note The degree of the zero polynomial is conventionally -1 or - (not defined in the textbook).

    Note The subset {0} is a subspace of dimension 0, since its basis is { } or which has dimension 0.

    • Any set containing 0 is dependent because c10=0 has a solution c1=1
    • So its basis cannot be {0} since a basis cannot be a linearly dependent set

    Thm (Lin Ind Set Basis Spanning Set)

    Thm 3.3 (Unique Representation)

    Let B be a set of non-zero vectors in a vector space V.

    Then B is a basis for V iff each vector v in V can be uniquely expressed as v=r1b1+...+rkbk where riR and biB.

    Proof for Thm 3.3

    When B is finite:

    () Suppose B is a basis (so it is lin ind) and  v that can be expressed the following 2 ways:

    v=r1b1+...+rkbkv=s1b1+...+skbk0=(r1s1)b1+...+(rksk)bk

    B is linearly independent, so r1s1=0,...,rksk=0r1=s1,...,rk=sk v has unique representation.

    () Suppose every vector can be uniquely expressed in terms of B. This implies V = span(B).

    0 can be uniquely expressed in terms of B, so r1b1+...+rkbk=0r1=...=rk=0 B is lin ind.

    B spans V and is linearly independent, so B must be a basis.

    When B is infinite:

    Def. Finitely Generated

    V is finitely generated if it is the span of a finite set, i.e. if V=sp(X) where X is finite.

    Q Which of these vector spaces is finitely generated?

    1. The vector space of all functions RR?
    2. The vector space of all polynomials.
    3. The vector space of all polynomials of degree less than or equal to n.
    4. The vector space of all 2×2 matrices.

    A 3 is finitely generated as it is the span of the finite set {1,x2,...,xn}

    4 is finitely generated as it is the span of the finite set of 2x2 matrices Eij where the ij-th entry = 1 and all other = 0

    Thm (Shortcut for proving basis)

    Let B be a set of m vectors from V. If |B| = dim(V), then showing one of (B is lin ind, B spans V) will suffice.

    Proof (If dim(V)=m, then any lin ind B is a basis of V.)

    B is a basis for V if it spans V and is lin ind. Lin ind is given, so just need to show it spans V.

    Suppose that B does not span V. Then  vV,vsp(B)dim(B{v})=m+1.

    This contradicts the fact that any set with >m elements cannot be lin ind in a vector space of dimension m.

    Proof (If dim(V)=m, then any B spanning V is a basis of V.)

    B is a basis for V if it spans V and is lin ind. We know B spans V, so just need to show lin ind.

    Suppose that B is not lin ind. Then at least 1 of its elements can be written as a lin combo of the other elements.

    Delete that element to get B, which would have the same span as B. B' would have m-1 elements.

    Continue deleting until lin ind. Since the set has the same span as B, it must be a basis for V.

    However, it would have <m elements, which contradicts dim(V)=m.

    Q Let U={p(x)P2:p(1)=0}. Show U is a subspace of P2 and find a basis for U.

    A Any p(x)U satisfies p(1)=0

    Consider p(x)=ax2+bx+c, then a(1)2+b(1)+c=0a+b+c=0{a=stb=sc=t where s,tR

    Plugging in a, b, c, we have p(x)=(st)x2+sx+t

    Then U={(st)x2+sx+t:s,tR}={s(x2+x)+t(x2+1):s,tR}=sp({x2+x,x2+1})

    By Span Theorem, U is a subspace of P2.

    {x2+x,x2+1} is linearly independent, so it is a basis of U and dim(U)=2.

    W3 (3.3, 3.4 pg. 213-216)

    Recall: If T:RnRm is a linear transformation, then T is induced by a matrix Amn=[T(e1)|...|T(en)] where {e1,...,en} is the standard basis of Rn. Then for xRn we have T(x)=Ax.

    Ordered Bases

    We can give an order to every basis. Note: To emphasize the basis is ordered, we use ( ) instead of { }.

    E.g. R3 has 6 ordered bases using {e1,e2,e3} = {[100],[010],[001]}

    They are: (e1,e2,e3),(e1,e3,e2),(e2,e1,e3),(e2,e3,e1),(e3,e1,e2),(e3,e2,e2)

    Def. Coordinate Vector

    Let B=(b1,...,bn) be an ordered basis for a vector space V.

    Suppose v=c1b1+...+cnbn, then the coordinate vector of v relative to the basis B is defined as vB=[c1cn]

    By the unique representation theorem, this vector is well defined.

    E.g. Let v=2x33x+1P2

    If B=(1,x,x2), then vB=[132]

    If B=(x,2x2,1), then vB=[321]

    Thm (Coordinatization of Vector Spaces)

    Any finite dimensional vector space V with dimension n can be coordinatized to look like Rn.

    We choose an ordered basis (b1,...,bn).

    Since each vector vV has a unique coordinate vector relative to B, there is a 1-1 correspondence between objects in V and vectors in Rn

    We may now work over Rn instead of V

    To see this, we verify that vector space operations are mirrored by operations on coordinate vectors in Rn

    Proof

    We show that for v,wV,tR, vector space ops = Rn ops:

    (1) Show that (v+w)B=vB+wB

    Suppose (by unique representation theorem) we have vectors v and w:

    v=r1b1+...+rnbnvB=(r1,...,rn)

    w=s1b1+...+snbnwB=(s1,...,sn)

    Then (v+w)B=(r1b1+...+rnbn+s1b1+...+snbn)B

    =((r1+s1)b1+...+(rn+sn)bn)B

    =[r1+s1rn+sn]=[r1rn]+[s1sn]=vB+wB

    (2) Show that (tv)B=tvB

    (tv)B=(t(r1b1+...+rnbn))B=(tr1b1+...+trnbn)B=[tr1,...,trn]=t[r1,...,rn]=tvB

    Def. Isomorphic

    When we rename vectors in V using coordinates, the vector space of coordinates (i.e. Rn) has the same structure as V.

    If V appears structurally identical to W, we say V and W are isomorphic, or VW.

    E.g. PnRn+1, Mm,nPmn1Rmn

    Thm (Dimension Determines Structure)

    If V is a vector space over R and dim(V)=n, then V is isomorphic to Rn

    Hence, to know the structure of a vector space (no matter how complicated , are), we only need to know its dimension - i.e. V and W are isomorphic dim(V) = dim(W)

    This is contrary to other mathematical structures like groups, rings, etc.

    Def. Linear Transformation

    Let V and W be vector spaces, then the function T:VW is a linear transformation if

    1. T preserves addition

      For v1,v2V, we need T(v1+v2)in V=T(v1)+T(v2)in W

    2. T preserves scalar multiplication

      For all VV,kR, we need T(kv)=kT(v)

    It is clear to see that T preserves linear combinations: T(a1v1+...+akvk)=a1T(v1)+...+akT(vk)

    Fact 2 lin transf are the same if they have the same value at each basis vector bi

    Proof Let T and T' be 2 linear transformations such that T(bi)=T(bi)  biB

    Let vV, then  b1,...,bkB and r1,..,rkR such that v=r1b1+...+rkbk

    T(v)=T(r1b1+...+rkbk)=r1T(b1)+...+rkT(bk)=r1T(b1)+...+rkT(bk)=T(r1b1+...+rkbk)=T(v)

    Thus T,T are the same transformation

    Other facts:

    1. T preserves zero vector

      T(0V)=0W

      Proof T(0)=r=0T(00)=0T(0)=0

    2. T preserves additive inverse (and thus substraction)

      T(v)=T(v)

    E.g. T:Mn,nR defined T(A)=tr(A) is a linear transformation

    Recall tr(A)=i=1naii

    1. T preserves addition

      T(A+B)=tr(A+B)=tr(A)+tr(B)=T(A)+T(B)

    2. T preserves scalar mult

      T(kA)=tr(kA)=k tr(A)=k T(A)

    E.g. T:Mn,nR,n>1 defined T(A)=detA is not a linear transformation

    1. T preserves addition

      Consider A = B = I

      T(A)+T(B)=T(I)+T(I)=detI+detI=1+1=2

      T(A+B)=T(I+I)=det(I+I)=det(2I)=2n

      Recall Determinant of a diagonal matrix is the product of the diagonal entries

      The above are not equal if n > 1

    2. T preserves scalar mult - no need to check

    E.g. Derivatives and integrals are linear transformations

    T(f)=f

    • T(f+g)=(f+g)=f+g=T(f)+T(g)
    • T(rf)=(rf)=r(f)=rT(f)

    T(f)=abf(x)dx

    • T(f+g)=ab[f(x)+g(x)]dx=abf(x)dx+abg(x)dx=T(f)+T(g)
    • T(rf)=abrf(x)dx=rabf(x)dx=rT(f)

    Def. Domain, Codomain

    Let V and W be vector spaces and T:VW be a linear transformation.

    Then V is the domain of T, W is the codomain of T.

    Def. Image, Inverse Image

    The image of T (or image of V under T, in blue) is a subspace of W: (T)={T(v):vV}=T[V]

    Labelling (T) as W', then the inverse image of W' under T is a subspace of V: {v:T(v)W}=T1[W]

    kernel_image

    Def. Range, Kernel / Null Space

    T[V] is the range of T (also image of V under T).

    The kernel of T (or inverse image of 0W under T, in green) is a subspace of V: ker(T)={v:T(v)=0W}=T1[{0W}]

    It is clear to see that ker(T) is solution set of the homogeneous transformation equation T(v)=0W

    Thm (Rank-Nullity)

    dim(V)=rank(T)+nullity(T)=dim((T))+dim(ker(T))

    Intuition: each dimension (or component) in V either gets preserved or compressed to 0W

    Proof

    Let {v1,...,vk} be a basis for ker(T) so dim(ker(T))=k

    Extend it to a basis for V:{v1,...,vk,w1,...,wm} so dim(V)=k+m

    We know dim(V)=k+m,dim(ker(T))=k, just need to prove dim((T))=m

    Do so by finding a basis for the image:

    By Unique Rep Thm, we can write any vV as v=a1v1+...+akvk+b1w1+...+bmwm,

    so (T)={T(v):vV}=sp({T(v1),...,T(vk)T(vi)=0W,T(w1),...,T(wm)})=sp({T(w1),...,T(wm)})

    WTS {T(w1),...,T(wm)} is lin ind: set up the dependence relation λ1T(w1)+...+λmT(wm)=0W

    T(λ1w1+...+λmwm)=0W since T is linear

    λ1w1+...+λmwmker(T)

    λ1w1+...+λmwm=μ1v1+...+μkvk since {v1,...,vk} is a basis for ker(T)

    λ1w1+...+λmwm+(μ1)v1+...+(μk)vk=0V

    λ1=...=λm=μ1=...=μk=0 since {v1,...,vk,w1,...,wm} is lin ind (it is a basis)

    So {T(w1),...,T(wm)} is lin ind, which means it is a basis for the image.

    Thus dim((T))=m,dim(V)dim(ker(T))=k

    Technique to Find Bases for Image(T)

    1. find a basis for ker(T)
    2. extend it to a basis for V
    3. apply T to what's been added to get a basis for (T)

    E.g. Let A=[0110] and T:M2,2M2,2 be a lin transf defined by T(X)=XAAX  XM2,2

    Find a basis for (T) using the above technique.

    1. Find a basis for ker(T)

      X=[abcd]ker(T)[abcd][0110][0110][abcd]=[0000][bcaddacb]=[0000]

      {bc=0ad=0da=0cb=0[01100100101001001100][10010011000000000000]{c=sd=ta=tb=s

      ker(T)={[tsst]:s,tR}={s[0110]+t[1001]:s,tR}=sp([0110],[1001])

    2. Extend it to a basis for V

      Extend by adding elements of the standard basis for M2,2

      {[0110],[1001],[1000],[0100]} (This is not the only way)

    3. Apply T to what's been added to get a basis for (T)

      {T([1000]),T([0100])}={[0101],[1001]}

    E.g. Let T:P4P4 be the linear transformation defined by T(p)=xp(x)p(x). Find a basis for the image of T and a basis for the kernel of T.

    Method 1: finding a matrix rep rel. to ordered bases and observing the null/column space

    Let B=B=(1,x,x2,x3,x4) be an ordered basis for P4.

    T(1)=x(0)1=1

    T(x)=x(1)x=0

    T(x2)=x(2x)x2=x2

    T(x3)=x(3x2)x3=2x3

    T(x4)=x(4x3)x4=3x4

    Then the matrix A for T relative to B,B is

    A=[T(1)BT(x)BT(x2)BT(x3)BT(x4)B]=[1000000000001000002000003]

    A basis for the null space of A is {[0,1,0,0,0]}, thus, a basis for ker(T) is {x}.

    Note: dim(ker(T)) = dim(null space(A))

    A basis for the column space of A is {[1,0,0,0,0],[0,0,1,0,0],[0,0,0,2,0],[0,0,0,0,3]}, thus, a basis for im(T) is {1,x2,2x3,3x4}

    Method 2: finding a basis for the kernel of T and extending it to get a basis for the image of T

    ker(T)=T1[{0W}] so to find the kernel, we need to find p such that T(p)=0

    Using p(x)=ax4+bx3+cx2+dx+e, T(p)=xp(x)p(x)=0x(4ax3+3bx2+2cx+d)(ax4+bx3+cx2+dx+e)=0

    Expand and simplify to obtain 3ax4+2bx3+cx2e=0a=b=c=e=0, d=k,kR

    (Or don't expand and obtain the corresponding system [4a=a,3b=b,2c=c,d=d,0=e])

    Thus, p=kxker(T)={kx:kR}=sp(x){x} is a basis for ker(T).

    Now extend this to a basis for P4 — add 1,x2,x3,x4 to the set.

    This implies that a basis for im(T) is {T(1),T(x2),T(x3),T(x4)}={1,x2,2x3,3x4}

    Def. Row Space

    Motivation Suppose U is a subspace of Rn with a basis of {v1,v2,...,vm}.
    What operations can be performed on this basis while preserving its span and linear independence?

    If {v1,v2,...,vm} are the rows of a matrix, these are simply elementary row operations!

    If A is a m x n matrix, then the row space of A is the subspace of Rn spanned by its rows, denoted row(A)

    Facts

    Thm The non-zero rows of any REF is a basis for row(A)

    E.g. Find a basis for U=sp{[10],[11],[22]} using row space

    A=[101122]row(A)=sp{[10]T,[11]T,[22]T}

    A=[101122][101100][100100][100100]

    By the theorem, row(A) has a basis of {[1,0],[0,1]}U has a basis of {[10],[01]}

    E.g. Find dim(row(A)) if A=[11132211513117001113]

    A row reduces to [11132011130000000000]{[1,1,1,3,2],[0,1,1,1,3]} is a basis

    So dim(row(A)) = 2

    Def. Column Space

    If A is a m x n matrix, then the column space of A is the subspace of Rm spanned by its columns, denoted col(A);

    col(A) = row(AT), so a basis of col(A) can be computed by reducing AT to REF

    Note (A)=col(A)

    E.g. Find a basis for col(A) if A=[12120241103614100150]

    A row-reduces to [12030001500000100000]

    Since (col 2) = 2(col 1), and (col 4) = 3(col 1) + 5(col 3),

    a basis for col(A) is the other 3 columns: {[1230],[1111],[0010]}

    Note We can't use columns of the REF for a basis, since row operations mess up the vectors!

    Strategy for finding basis of row(A) and col(A)

    1. reduce A to REF = R
    2. A basis for row(A) is formed by rows of R containing leading 1's
    3. A basis for col(A) is formed by cols of A corresponding to cols of R with leading 1's

    Def. Rank

    The rank of a m x n matrix A is the number of leading 1's in its REF, and rank(A)min(m,n)

    rank(A)=dim(row(A))=dim(col(A))=dim((A))

    Both of the below have rank(A)=2=dim(row(A))=dim(col(A))

    (1) m=2,n=3:[100010]

    rank(A)=mcol(A)=Rm columns of A span Rm; rows of A are independent in Rn

    (2) m=3,n=2:[100100]

    rank(A)=nrow(A)=Rn rows of A span Rn; columns of A are independent in Rm

    E.g. Can a 5 x 6 matrix have linearly independent rows/cols?

    The matrix would reduce to [100000010000001000000100000010]

    Cols cannot be linearly independent since each column has 5 entries (so R5) but there are 6 columns. Only 5 are needed to span R5. The 6th column can be written as a lin combo of the first 5. The 6th variable can be anything.

    Rows can be linearly dependent or independent.

    Note The converse: if # rows > # columns, the rows must be lin dep, the cols may be lin dep.

    E.g. Let A be m x n with rank(A)=m, prove mn

    rank(A)n by definition, and rank(A)=m, so mn

    E.g. Let A be a 5 x 9 matrix, prove dim(null(A))4

    rank(A)5 by definition, and rank(A)=ndim(null(A))9dim(null(A))54dim(null(A))

    Note rank-nullity theorem was used here: for Am,n, rank(A)+nullity(A)=n

    W4 (remainder of 3.4)

    Thm 3.7 (Lin transf preserves subspaces)

    Let V and V' be vector spaces and let T:VV be a linear transformation.

    Proof (If S is a subspace of V, then T[S] is a subspace of V)

    Subspace test:

    1. WTS T[S] contains the zero vector

      Since S is a subspace, 0VST(0V)T[S]linearity of T0VT[S]

    2. WTS T(s1)+T(s2)T[S], where T(s1),T(s2)T[S] and so s1,s2S

      Since S is a subspace, s1+s2ST(s1)+T(s2)T[S]linearity of TT(s1+s2)T[S]

    3. WTS rT(s)T[S], where T(s)T[S] and so sS

      Since S is a subspace, rsST(rs)T[S]linearity of TrT(s)T[S]

    Proof (If S is a subspace of V, then T1[S] is a subspace of V)

    1. WTS T1[S] contains the zero vector

      Since S is a subspace, 0VST1(0V)T1[S]linearity of T10VT1[S]

    2. WTS v1+v2T1[S], where v1,v2T1[S], and so T(v1),T(v2)S

      Since S is a subspace, T(v1)+T(v2)ST1(T(v1)+T(v2))T1[S]linearity of Tv1+v2T1[S]

    3. WTS rvT1[S], where vT1[S] and so T(v)S

      Since S is a subspace, rT(v)ST1(rT(v))T1[S]linearity of TrvT1[S]

    Def. Onto & 1-1

    Let V and W be vector spaces and T:VW is a linear transformation

    Onto (surjective)1-1 (injective)
    for any wW, vV such that T(v)=wfor any v1,v2V, T(v1)=T(v2)v1=v2
    or v1v2T(v1)T(v2)
    all elements in W have been mapped to at least onceall elements in W have been mapped to at most once
      wW,T(v)=w has at least 1 solution  wW,T(v)=w has at most 1 solution
      wW,Av=w is consistent  wW,Av=w has a unique sol'n or is inconsistent
    A has a pivot in every rowA has a pivot in every column
    The columns of A span WThe columns of A are lin ind
    i.e. Ax=0 has only the trivial solution
    i.e. ker(T)={0V}
    T preserves spanning sets:
    Suppose V=sp{v1,...,vk}
    then W=sp{T(v1),...,T(vk)}
    T preserves lin ind:
    Suppose {v1,...,vk} is a lin ind subset of V,
    then {T(v1),...,T(vk)} is a lin ind subset of W.
    dim(range of T) = dim(W)dim(range of T) = dim(V)

    Note If T is both onto and 1-1, then it is bijective (and it must be an isomorphism, so it's invertible).

    For bijective T:VW,dim(V)=dim(W),T(v)=Av=w has a unique solution (i.e. A has a pivot in every row and column)

    Thm (Extra TB notes)

    Let T(p)=b for some particular vector in V.

    Then the solution set of T(x)=b is the set {p+h|hker(T)} (h is the solution to the homogeneous transf eq)

    This means:

    So 1-1 ker(T)=0V follows from this.

    E.g. Show that the solution of y4y=x2 is the set {p+h|hker(T)}

    (So T(y)=y4y , and b=x2 )

    • Want to find p such that T(p)=b, i.e. find y such that y4y=x2

      By inspection, try y=x24. Then we have 124y=4y12=0 which can't be true

      So kill off the 12 with y=x2124, then we have 124(x2124)=x2 which holds

      So we've found the solution: p(x)=y=x2124

    • The homogeneous transformation eq T(x)=0 in this case is y4y=0

      Its general solution is h(x)=c1e2x+c2e2x{e2x,e2x} is a basis for ker(T)hker(T)

    • The general solution for y4y=x2 is thus h(x)+p(x)=c1e2x+c2e2x+(x2124)

    Thm (Compositions of lin transf is a lin transf)

    Let T:UV and S:VW be linear transformations (U, V, W are vector spaces).

    Then ST:UW is also a linear transformation

    Proof Let x,yV,a,bR

    WTS ST(ax+by)=aST(x)+bST(y)

    ST(ax+by)=S(T(ax+by))=S(aT(x)+bT(y))=aS(T(x))+bS(T(y))=aST(x)+bST(y)

    Def. Invertible Transformation

    A linear transformation T:VW is invertible if there is a linear transformation T1:WV such that T1T=IV,TT1=IW, where IW is the identity transformation mapping vectors from W to W as IW(x)=x

    T1 is the inverse for T (using "the" because it is unique).

    Fact T is invertible T is an isomorphism

    Thm (T is invertible if A is invertible)

    Suppose that B={b1,,bn} is a basis for the vector space V, C={c1,,cn} is a basis for the vector space W, and T:VW is a linear transformation. Show that T is an invertible transformation iff the matrix representation [T]BC (or denoted by RB,C) of T wr.t. the bases B and C is an invertible matrix.

    Suppose T is invertible, then  T1:WV such that T1T=IV,TT1=IW.

    Let [T1]CB be the matrix representation of T1 with respect to the bases C and B.

    Since [T1]CB[T]BC=In and [T]BC[T1]CB=In, [T]BC is an invertible matrix.

    Assuming [T]BC is an invertible matrix, there must be a matrix A such that A[T]BC=In, [T]BCA=In

    Let A be the matrix rep of the linear transf. S:WV, i.e. [S]CB=A.

    Then ST=IV, and TS=IW (reads identity transf. on W), so S is an inverse for T, and T is invertible.

    Thm 3.8 (T is invertible iff 1-1 and onto)

    Proof Suppose T:VV

    (): invertibility implies 1-1 & onto

    Since T is invertible,  vV such that T1(v)=v for all vV. This means T must be onto V.

    v=(TT1)(v)=T(T1(v))=T(v), so v1v2T(v1)T(v2). Hence T must be 1-1.

    (): 1-1 & onto implies invertibility

    Since T is onto V,  vV such that T1(v)=v for all vV. Since T is 1-1, this v must be unique.

    Define T1:VV as T1(v)=v, where v is that unique vector satisfying T(v)=v

    Then (TT1)(v)=T(T1(v))=T(v)=vTT1=IV

    and (T1T)(v)=T1(T(v))=T1(v)=vT1T=IV

    Hence T is invertible.

    Remains to show that T1 is indeed a linear transformation:

    1. Preserves addition

      T1(v1+v2)=T1(T(v1)+T(v2))=T1(T(v1+v2))=v1+v2=T1(v1)+T1(v2)

    2. Preserves scalar mult

      T1(rv1)=T1(rT(v1))=T1(T(rv1))=rv1=rT1(v1)

    Def. Isomorphism

    A linear transformation T:VW is an isomorphism if it is 1-1 and onto.

    E.g. Prove M2,2P3

    Construct a linear transformation T:M2,2P3 that is 1-1 and onto:

    Consider T([abcd])=a+bx+cx2+dx3

    1. Show linearity (T preserves addition and scalar mult)

      T(r[abcd]+s[efgh])=T([ra+serb+sfrc+sgrd+sh])=(ra+se)+(rb+sf)x+(rc+sg)x2+(rd+sh)x3=r(a+bx+cx2+dx3)+s(e+fx+gx2+hx3)=rT([abcd])+sT([efgh])

    2. Show 1-1 (by showing ker(T)=0)

      T(A)=0T([abcd])=0+0x+0x2+0x3a+bx+cx2+dx3=0+0x+0x2+0x3a=b=c=d=0ker(T)={[0000]}

    3. Show onto

      For any polynomial p(x)=q+rx+sx2+tx3P3, we can find a matrix in M2,2 that maps to it, namely

      A=[qrst]

    Thm (Iso proof shortcut if same dim)

    Let V, W be finite dimensional vector spaces and T:VW a linear transformation.

    If dimV=dimW, then T is an isomorphism iff T is OR(onto, 1-1).

    Prove using the rank nullity thm: dim((T))+dim(ker(T))=dim(V)

    same dim & 1-1 invertible

    If dim(W)=dim(V)=n, and T is 1-1, then ker(T)={0}dim(ker(T))=0

    By the rank nullity thm, dim((T))+0=n.

    And since (T) is a subspace of W which also has dimension n, we have that (T)=W. Hence, T is onto.

    Since T is both 1-1 and onto, it must be invertible.

    same dim & onto invertible

    If dim(W)=dim(V)=n, and T is onto, then (T)=Wdim((T))=n

    By the rank nullity thm, n+dim(ker(T))=ndim(ker(T))=0ker(T)={0}T is 1-1.

    Since T is onto and 1-1, it must be invertible.

    Thm 3.9 (Coordinatization is an isomorphism)

    Let V be a finite dimensional vector space with ordered basis B=(b1,...,bn)

    Then T:VRn defined by T(v)=vB is an isomorphism.

    Proof

    T is a lin transf since it preserves addition and scalar mult. as proved in W3: Coordinatization of Vector Spaces

    T is an isomorphism:

    1. T is 1-1 since the coordinate vector vB uniquely determines v
    2. T is onto since the range of T is all of Rn

    Note The isomorphism of V with Rn is not unique, since there is a corresponding isomorphism for each choice of an ordered basis of V.

    W5 (7.1, 7.2)

    Def. Standard Matrix Rep of Lin Transf

    Recall concepts for T:RnRm

    Q Find the standard matrix rep of T([x1,x2,x3])=[x1x2+3x3,x1+x2+x3,x1]

    A For T:RnRm, the standard matrix rep is given by Am×n=[T(e1)|...|T(en)]=T(Im×n)

    Since T:R3R3, A=[113111100]

    Shortcut for finding A: instead of calculating each col vector by transforming ej,
    just write out the coefficients of the transformed input as row vectors

    Now, how do we find the matrix rep of T:VW, where V,W are non-Euclidean vector spaces?

    E.g. Define τ:M2,2P3 by τ([abcd])=a+(a+c)x+(bc)x2+dx3

    To represent τ with a matrix, we start with an ordered basis for each of M2,2 and P3

    For M2,2:B=([1000],[0100],[0010],[0001])

    For P3:C=(1,x,x2,x3)

    Since it is easier to work over Rn, we use the following coord. transformations:

    R:M2,2R4 defined by R([abcd])=[abcd]

    S:P3R4 defined by S(a+bx+cx2+dx3)=[abcd]

    T:R4R4 defined by T([abcd])=[aa+cbcd]

    This way, we can write T as a composition of linear transformations: T=SτR1

    (since R4R4R4M2,2P3R4, illustrated below)

    Since T is a composition of lin transf, it must be linear itself, so find the standard matrix for T:

    A=T([1000010000100001])=[1000101001100001] is said to be the matrix of τ with respect to B and C.

    Also, A is invertible (it has a non-zero det) T is invertible T is an isomorphism.

    τ=S1TRτ is also an isomorphism.

    Matrix Rep Notation

    Matrix rep of T relative to B and B can be denoted as RB,B, RB,B(T), or [T]BB

    The following denote the matrix rep of T inverse

    The following denote S composed with T illustrated above

    The following denote the change of basis matrix CB,B illustrated below

    Def. Matrix of T Rel. to Basis B

    Let V be a finite dimensional vector space with ordered bases B=(b1,,bn)

    Let T:VV be a lin transf, then RB=[T(b1)B...T(bn)B] is the matrix of T relative to basis B

    T(v)B=RBvB

    Def. Matrix of T Rel. to Bases B and B'

    Let V and V' be finite dimensional vector spaces with ordered bases B=(b1,,bn) and B=(b1,...,bm).

    Let T:VV be a lin transf and T¯:RnRm be the lin transf such that T¯(vB)=T(v)B.

    I.e. T¯ transforms vB (coord. of v rel. to B) into the coord. of transformed v rel. to B'

    Then the matrix rep of T relative to B and B is the m x n matrix A=[ T(b1)B  ... T(bn)B ]

    I.e. A is the coordinates of the transformed basis vectors (of B) relative to B

    And so T(v)B=A(vB)  vV

    I.e. the coord. of any transformed vector can be obtained by:
    multiplying A (coord. of transformed basis vectors rel to B') with its coord. relative to B

    Note A1 is the matrix rep of T1 relative to B,B.

    Prove: j-th column of A = T(bj)B

    Since the coord vector of v=a1b1+...+anbn rel. to B is given by vB=[a1an]

    the coord vector of bj=0b1+...+1bj+...+0bn rel. to B is given by [bj]B=[010]=ej so [bj]B=ej

    T(v)B=A(vB)T(bj)B=A[bj]B=Aej=[a11...a1nam1...amn][010]= j-th column of A

    E.g. Find RB,B where B=B=(sinxcosx,sin2x,cos2x), and use it to compute f(x), given

    f(x)=3sinxcosx5sin2x+7cos2x

    RB,B=[T(sinxcosx)BT(sin2x)BT(cos2x)B]=[(sin2x+cos2x)B(2sinxcosx)B(2sinxcosx)B]=[022100100]

    T(v)B=RB,BvB

    [f(x)]B=RB,B[f(x)]B=[022100100][357]=[2433]

    f(x)=24sinxcosx3sin2x+3cos2x

    E.g. Consider T:P3M2,2 defined by T(ax3+bx2+cx+d)=[2a2b2b3c2a2b+d3c+d]

    Let B=(x3,x2,x,1) be an ordered basis for P3,

    and B=([1000],[0100],[0010],[0001]) an ordered basis for M2,2

    Find RB,B(T),RB,B(T1),T1([pqrs])

    RB,B(T)=[T(x3)BT(x2)BT(x)BT(1)B]=[[2020]B[2220]B[0303]B[0011]B]=[2200023022010031]

    RB,B(T1)=[2200023022010031]1=[11/21/21/21/21/21/21/21/301/31/31010]

    T1(v)B=RB,B(T1)(vB)

    T1([pqrs])B=RB,B(T1)[pqrs]=[p12q12r12s12p12q12r+12s13p+13r13sp+r]T

    T1([pqrs])=(p12q12r12s)x3+(12p12q12r+12s)x2+(13p+13r13s)x+(p+r)

    Def. Change of basis matrix

    If we have a vector space V with basis B, we can find a new basis B' using the matrix CB,B=[[b1]B...[bn]B] and the change of basis formula vB=CB,BvB

    I.e. (coord vec of v in new basis) = (change of basis matrix from B to B') (coord vec of v in old basis)

    Note Recall T(v)B=AvB from thm 3.10. Note its similarity to the change of basis formula.

    Note CB,B=CB,B1

    E.g. Find the change of basis matrix for B=(x21,x2+1,x2+2x+1),B=(x2,x,1)

    CB,B=[(x21)B(x2+1)B(x2+2x+1)B]=[111002111]

    If V=Rn, then row reduce the matrix [new basis vectors | old basis vectors] to find CB,B:

    [b1...bnb1...bn][ICB,B]

    E.g. Let V=R3. Find CB,B where B=(e1,e2,e3) and B=([1,1,0],[2,0,1],[1,1,0]).

    Also find vB where v=[234]

    [121100101010010001][1001/21/210100010011/21/21]

    Note: CB,B is essentially [ei]B since for i = 1, [100]=12[110]+0[201]+12[110]

    Then vB=CB,BvB=[1/21/210011/21/21][234]=[3/244/3]

    Motivation for next topic

    Consider T:P2P2 where T(P)=p is the derivative

    If B=(1,x,x2), then differentiate to get RB=[010002000]

    If B=(x+1,x1,2x2), then differentiate (and decompose): {T(x+1)=1=1/2(x+1)1/2(x1)+0(2x2)T(x1)=1=aboveT(2x2)=4x=2(x+1)+2(x1)+0(2x2)

    so RB=[1/21/221/21/22000]

    RB,RB use different bases, but represent the same transformation T. Their similarities are :

    These similarities are not a coincidence. Similar matrices have similar properties.

    Def. Similar Matrices

    Let A, B be square matrices, then A is similar to B if there is an invertible C such that B=C1AC

    A and B would have the same rank, determinant, eigenvalues, trace, and change of basis matrix.

    Note A and B are similar iff they are the matrix reps of the same linear transformation relative to different bases. (thm 7.1)

    Thm 7.1 (Basis Change in A Vector Space)

    If we have a finite dimensional vector space V with bases B,B, and a linear transformation T:VV, then the matrix reps for T, RB and RB must be similar, so RB=C1RBC where C=CB,B

    Rearranging, we have CB,BRB=RBCB,B which is illustrated below (right + down = down + right)

    Since CB,B=(CB,B)1, we have RB=CB,BRB(CB,B)1=(CB,B)1RBCB,B

    E.g. T:P2P2,T(p(x))=p(x1)

    Let B=(x2,x,1),B=(x,x+1,x21)

    Find RB,RB,C such that RB=C1RBC

    CB,B=[[b1]B[b2]B[b3]B]=[001110011]

    RB=[T(x2)BT(x)BT(1)B]=[(x1)B2(x1)B1B]shift x by 1=[(x22x+1)B(x1)B1B]=[100210111]

    RB=[T(x)BT(x+1)BT(x21)B]=[(x1)B(x)B((x1)21)B]shift x by 1=[(x1)B(x)B(x22x)B]=[213101001]{x1=2(x)1(x+1)+0(x21)x=1(x)+0(x+1)+0(x21)x22x=3(x)+1(x+1)+(x2+1)=C1RBC

    Shortcut to find Matrix Rep (Add basis E)

    In the above, if we let S=I, and let W have bases B and E . Then U=W and we have the following

    RB,E=CB,ERB,B

    Rearrange the above to get:

    E.g. Let V be P3 polynomials, T:P3P3 be a linear transf. defined by T(p(x))=x2p(x)

    Find RB if B=(1,1+x,1x2,1+x3)

    Let E be the standard basis (1,x,x2,x3), then RB,B=(CB,E)1RB,E

    CB,E=[1111010000100001]

    Apply T to basis and decompose/express as lin combo:

    T(1)=0=0(1)+0(1+x)+0(1x2)+0(1+x3)T(1+x)=0=0(1)+0(1+x)+0(1x2)+0(1+x3)T(1x2)=2x2=2(1)+0(1+x)+2(1x2)+0(1+x3)T(1+x3)=6x3=6(1)+0(1+x)+0(1x2)+6(1+x3)

    so RB,E=[0000000000200006]

    Recall [A|I][I|A1]the row ops essentially multiply both sides by A1

    [A|C][I|A1C]

    So to find RB,B=(CB,E)1RB,E, row reduce [CB,E|RB,E][I|(CB,E)1RB,E]=[I|RB,B]

    [11110000010000000010002000010006][10000026010000000010002000010006]

    We have found RB (and to find T(v), use RBvB=T(v)B)

    Thm (Basis Change between 2 Vector Spaces)

    For a linear T:VW, where V has ordered bases A,A, and W has ordered bases B,B, we can change coord from A,B A,B using

    Def. Eigen Values/Vectors

    Given a transformation, an eigen vector is a non-zero vector that only gets stretched/scaled by a factor (not morphed)

    Av=λvwhere λ is the scaling factor, called eigen valueAv=(λI)vSince LHS is matrix mult, turn RHS into matrix mult as well(AλI)v=0AλI transforms v to 0det(AλI)=0Since the determinant is the scaling factor for a volume under a transformation, it must = 0 here

    To obtain the eigen value, solve the equation that arises from det = 0 (last line), called the characteristic polynomial

    To obtain the eigen vectors, plug the eigen values back into the second to last line, and solve the system

    Let T:VV be linear, V be a vector space

    Fact If T:RnRn, then eigen values of T are eigen values of the standard matrix rep of T

    E.g. Consider vector space P2 and basis E=(1,x,x2).

    Find the eigen values and eigen vectors of T:P2P2 defined by {T(1)=3+2x+x2T(x)=2T(x2)=2x2

    Find matrix for T relative to E:

    RE=[320200102]

    Find eigen values by performing cofactor expansion down the 3rd column:

    det(REλI)=|3λ202λ0102λ|=(2λ)|3λ22λ|=(2λ)((3λ)(λ)4)=(2λ)(3λ+λ24)=(2λ)(λ+1)(λ4)

    EigenvaluesEigenvectors for P2Eigenvectors for T
    λ1=1[-3, 6, 1]p1(x)=3+6x+x2
    λ2=2[0, 0, 1]p2(x)=x2
    λ3=4[2, 1, 1]p3(x)=2+x+x2

    Note If the basis consists of eigenvectors for T, i.e., B=(3+6x+x2,x2,2+x+x2),
    then the matrix rep of T relative to B is RB=λI=[100020004]

    Def. Eigen Space

    If λ is an eigen value of A, then the eigen space of A , denoted Eλ, is the set of eigen vectors corresponding to λ plus the zero vector (which isn't categorized as an eigenvector, even though it has the properties of one).

    Note Eλ={x:(AλI)x=0}=null(AλI)=ker(AλI) so it is a subspace of n-space (either Rn or Cn).

    Def. Alg & Geo Multiplicities

    Algebraic multiplicity of λ = its multiplicity (as a root) in the characteristic polynomial

    Geometric multiplicity of λ = dim(Eλ)

    Def. Diagonalizable

    Def. An is diagonalizable if it is similar to a diagonal matrix D=P1APA=PDP1

    P is said to diagonalize An

    Note The diagonal entries of D are the eigenvalues λi, and the column vectors of P are the eigen vectors

    Thm An is diagonalizable

    An has n linearly independent eigen vectors (i.e. enough eigen vectors to form a basis for Rn)

    ​ sum of (alg mult of eigen values of A) = n, and each eigen value has geo mult = alg mult

    sum of (geo mult of eigen values of A) = n (so if  r distinct eigen values, then Rn=Eλ1...Eλr

    where means direct sum, e.g. RR=R2)

    E.g. Consider T:P2P2 defined by T(p)=p, e.g. T(ax2+bx+c)=2ax+b. Is T diagonalizable?

    Let B=(1,x,x2) be a standard basis of P2

    Then [T]BB=RB,B=[010002000]

    Does there exist 3 lin ind. eigen vectors for the matrix rep of T?

    The characteristic polynomial of [T]BB is p(λ)=det([T]BλI)=|λ100λλ00λ|=(λ)3=λ3

    So λ=0 is an eigenvalue for [T]BB with algebraic multiplicity 3 (λ1=λ2=λ3=0)

    And [T]BB0I=0[010002000][010001000] eigenvectors are r[100],r0

    So the eigenspace Eλ={r[100]:rR}=sp([100]) and the geometric multiplicity of eigenvalue 0 is 1

    Since there isn't enough eigenvectors, the sum of dim(Eλ)dim(vector space), so T is not diagonalizable.

    Note If T is diagonalizable, then it is easy to compute powers of T, e.g. T(T(V)) by finding [T]B which is the (diagonal) matrix rep of T relative to the basis of eigen vectors (denoted B') and taking the appropriate power of the matrix .

    W6 (6.1)

    Def. Orthogonal

    Let x=[x1xn] and y=[y1yn] be vectors.

    Their dot product is xy=x1y1+...+xnyn=yTx=xycos(θ) where θ is the angle between x and y.

    Since xy=0 when cosθ=0θ=π2=90°, they are said to be orthogonal (perpendicular) if xy=0

    The length of x is x=x12+...+xn2 and x2=xx

    A unit vector in the direction of v is obtained by dividing by its length: u=1vv

    Def. Orthogonal Set

    The set {x1,...,xk}Rn is an orthogonal set if each xi is not the zero vector and every pair of distinct vectors is orthogonal.

    Note 0 not included so that certain properties hold (e.g. orthogonal sets are lin ind.)

    Proof Orthogonal sets are linearly independent.

    WTS c1x1+...+ckxk=0 only has the trivial solution c1=...=ck=0

    Take dot prod of eq'n with xi,1ik:

    xi(c1x1+...+ckxk)=xi0c1(xix1)=0+c2(xix2)=0+...+ci(xixi)0+...+ck(xixk)=0=0ci=0

    Def. Orthonormal Set

    A set is orthonormal if it is orthogonal and xi=1  xi

    E.g. {12[1111],12[1111],12[1111]}

    Each pair has dot prod = 0 and each vector has length 1, so this set is orthonormal subset of R4

    Def. Normalize

    To normalize an orthogonal set, divide each element by its length to turn it into an orthonormal set

    Def. Orthogonal Complement

    Let W be a subspace of Rn. The orthogonal component of W is W={xRn:xw=0  wW}

    E.g. If W is a plane through the origin in R3, then W is the line through the origin perpendicular to the plane

    Facts Let W be a subspace of Rn

    Thm (Ortho Complement of Row/Col Space )

    Let A be an m x n matrix, then

    1. (row(A))=null(A)
    2. (col(A))=null(AT)

    Proof Let xRn

    1. x is in (row(A))x is orthogonal to every row of A Ax=0xnull(A)
    2. x is in (row(AT)col(A))x is orthogonal to every row of ATATx=0xnull(AT)

    Technique to Find Ortho Complement

    1. Find a matrix A whose rows are vectors that span W
    2. Find null(A)=W

    E.g. Find U if U=sp({[1000],[0110],[0201]})

    x=[abcd]U[abcd]ui=0 for i = 1, 2, 3. Solving the 3 equations, we have {a=0b+c=02b+d=0

    Solving the 3 equations is equivalent to just writing the ui's in a matrix: [abcd100000110002010]

    which row reduces to [1000001100001120]x=[0t/2t/2t],tR

    so U={x=[0t/2t/2t]:tR}=sp{[0112]s:sR}

    Check that [abcd]ui=0 and that dim(U)=dim(V)dim(U)

    W8 (6.2 - 6.4)

    Def. Projections (Textbook Def)

    Let b be a vector in Rn and W a subspace of Rn. Let b=bW+bW, then bW is the projection of b onto W

    Steps to find projection of b onto W

    1. Select a basis {v1,v2,...,vk} for W
    2. Find a basis {vk+1,vk+2,...,vn} for W
    3. Find coord. vector r=[r1rn] of b relative to basis {v1,...,vn} (proof below) so that b=r1v1+...+rnvn
    4. Then bw=r1v1+...+rkvk

    Proof If W is a subspace in Rn, {w1,...,wk} is a basis for W, and {wk+1,...,wn} is a basis for W, show that {w1,...,wk,wk+1,...,wn} is a basis for Rn.

    NTS it spans Rn (which it clearly does), and is linearly independent.

    Set up the dependence relation i=1naiwi=0

    Suppose v=a1w1+...+akwkvW

    By the dep. rel, we also have that v=(ak+1wk+1+...+anwn)vW

    Since vW,W, v=0 (since vv=0)

    Thus, a1w1+...+akwk=0 and (ak+1wk+1+...+anwn)=0

    Since {wk+1,...,wn} and {wk+1,...,wn} are linearly independent, a1=...=an=0

    {w1,...,wk,wk+1,...,wn} is lin ind.

    Note If instead, {w1,...,wk} is an orthonormal basis for W, and {wk+1,...,wn} is an orthonormal basis for W, then {w1,...,wk,wk+1,...,wn} is an orthonormal basis for Rn. Thus, every vRn can be written uniquely in terms of wi as v=(vw1)w1+...+(vwk)wkprojWv+(vwk+1)wk+1+...+(vwn)wnprojWv

    Proof (that coefficients are ai=vwi)

    If v=a1w1+...+anwn, then vwi=(a1w1+...+anwn)wi=a1w1wj0+...+aiwiwi1+...+anwnwi0=ai

    Def. Projections (Alternative Def)

    Let W be a subspace of Rn with an orthogonal basis{v1,...,vk}. Then the projection of bRn on W is projWb=projv1b+...+projvkb or bW=bv1v1v1v1+...+bvkvkvkvk

    Properties Let W be a subspace of Rn and vRn. Then

    E.g. Let W=sp([011],[011]). Find the vector in W closest to x=[232]

    Since W is the span of an orthogonal set, the answer is projWx by the projection theorem.

    xW=xw1w1w1w1+xw2w2w2w2=52[011]+12[011]=[032]

    Observe that the projWx is just x with its first value replaced with 0. This makes sense since W is the span of 2 vectors with x = 0, so it is like projecting onto the y-z plane. Hence another (simpler) way to approach this problem would be to choose the basis as the standard basis of the y-z plane {[010],[001]}

    Def. Gram Schmidt Algorithm

    Every subspace W of Rn has an orthonormal basis.

    Let {x1,...,xk} be a basis for W. The Gram Schmidt algo generates an orthogonal basis {v1,...,vk} for W.

    v1=x1

    v2=x2x2v1v1v1v1perp. to v1

    v3=x3x3v1v1v1v1x3v2v2v2v2perp. to v1,v2

    vk=xkxkv1v1v1v1...xkvk1vk1vk1vk1perp. to v1,...,vk

    Illustration for R2:

    For convenience, we can scale vi to remove fractions during the algorithm.

    To get an orthonormal basis, we divide each vector by its length at the end.

    Def. Orthogonal Matrix

    A square matrix Q is said to be orthogonal if any of the following is true

    Note If A is not square, then ATA=IAAT. This chain of equality only holds if A is square. (They do have the same non-zero eigen values however.)

    Properties

    Thm (Relation to Change of Basis)

    If B={w1,...,wn} is an orthonormal basis of Rn,

    then the change of basis matrix rel. to E is [I]BE=[w1...wn], which is an orthogonal matrix.

    So ([I]BE)1=([I]BE)T(Q1=QT)

    and vB=[I]BEQ vEv=vE(use Qv = v, proves Q preserves length)

    E.g. Let (a1,a2,a3,a4) be an ordered orthonormal basis for R4, and let [2,1,4,3] be the coordinate vector of a vector b in R4 relative to this basis. Find b.

    Let ϵ be the standard basis for R4 and B=(a1,a2,a3,a4), then [2,1,4,3]=[b]B

    b=[b]ε=[I]Bε[b]B=[b]B=[2,1,4,3]=30

    Def. Orthogonal Transformation

    A linear transformation T:RnRn is orthogonal if T(v)T(w)=vw  v,wRn

    (T is orthogonal its matrix is orthogonal)

    Def. Projection Matrix

    Let W be a subspace of Rn, then T:RnRn defined by T(x)=projWx is a linear transformation. The standard matrix for T is called the projection matrix for the subspace W.

    Steps (to find a projection matrix for subspace W)

    1. form a matrix using vectors from W's basis: A=[w1...wk]n×k so that W is the column space of A
    2. then P=A(ATA)1AT is the projection matrix for the subspace W and projWx=Px

    If {w1,...,wk} is an orthonormal basis, P=AAT

    E.g. Let W=sp([131313],[12120]). Find the projection matrix for W and projWx

    Let A=[13121312130]. Note that the columns are orthonormal, so A is orthogonal, i.e. ATA=I

    Then P=A(ATAI)1AT=AAT=[561613165613131313] and projWx=[P][x1x2x3]=16[5x1+x2+2x3x1+5x22x32x12x2+2x3]

    Proof (for P=A(ATA)1AT)

    Note that W = col(A), so all vectors in W can be written as Ax=[w1...wk][x1xk]=ixiwi

    projWbWprojWbcol(A)

     x^Rk such that projWb=Ax^

    bprojWb=bAx^ is perpendicular to vectors in W, which all have form Ax

    (bAx^)(Ax)=0

    (Ax)T(bAx^)=0since xy=yTx

    (xT)(ATbATAx^)=0since (Ax)T=xTAT

    (ATbATAx^)x=0write as dot product

    ATbATAx^=0can now solve for x^

    x^=(ATA)1ATbsince rank(ATA)=rank(A)=k(ATA)k×k has full rank so must be invertible

    projWb=Ax^=A(ATA)1ATPb

    Properties

    Every n x n matrix that is both symmetric and idempotent is a projection matrix.

    E.g. Find all invertible projection matrices.

    Since projection matrices are idempotent, P2=P.

    If P is invertible, then multiplying both sides of P2=P by P1 gives P=I.

    W9 (9.1, 9.2)

    Def. Least Squares

    Let A be an m x n matrix. Ax=b could have no solution, i.e. if b is not in col(A).

    Then the best "approximate solution" to Ax=b is the least squares solution x^, where bAx^bAx  xRn

    x^ minimizes error e=Ax^b2, which is the sum of squares of the errors in the m equations (when mn)

    If no solution, then multiply Ax=b by AT to get ATAx^=ATb, and solve for x^=(ATA)1ATb.

    Justification Look for x^ for which Ax^W=col(A).

    (Ax^b)col(A)(Ax^b)(col(A))=null(AT)={xRn:ATx=0}

    AT(Ax^b)=0ATAx^=ATb

    E.g. Find the line closest to points (0, 6) (1, 0) and (2, 0)

    y = mx + b {0m+b=61m+b=02m+b=0 so Ax=b[011121][mb]=[600]

    ATAx^=ATbx^=(ATA)1ATb=([012111][011121])1[012111][600]=[35]

    So y = -3x + 5 is the line of best fit

    Note ATA is invertible by the following thm.

    Thm (Invertibility, Lin Ind, and Uniqueness)

    Let A be an m x n matrix and bRm. Then the following are equivalent:

    Def. Symmetric Matrices

    A symmetric matrix A=AT always has real eigen values and is always diagonalizable (by a real orthogonal matrix).

    Properties

    Proof Let λC be an eigen value and A be a real matrix.

    Then Ax=λx for some x0

    Since A is real, A=A=AT

    LHS: xTAx=xTλx=λ(xTx)

    RHS: xTATx=(Ax)Tx=(λx)Tx=λ(xTx)

    Equating LHS and RHS, we have that λ=λλ is real

    Def. Orthogonally Diagonalizable

    A square matrix A is orthogonally diagonalizable if there is an orthogonal matrix Q and a diagonal matrix D such that A=QDQTQTAQ=D

    Note For a real n x n matrix A, symmetric orthogonally diagonalizable

    E.g. [1102] is diagonalizable but not orthogonally diagonalizable since it is not symmetric.

    Steps to orthogonally diagonalize a matrix A:

    1. Find the eigen values and eigen vectors
    2. Form a basis with the eigen vectors and normalize
    3. Form Q with the normalized eigen vectors as columns

    E.g. Diagonalize A=[1222] using an orthogonal matrix.

    The eigen values are λ1=3 with eigen vector v1=[12] and λ2=2 with eigen vector v2=[21]

    A=PDP1 with P=[v1|v2]=[1221] and D=[3002]regular diagonalization

    Note that {v1,v2} is an orthogonal set, so if we normalize it to {[1525][2515]}, then we can use

    A=QDQ1 with Q=[15252515] and D defined above. orthogonal diagonalization

    Since Q is orthogonal, Q1=QT, so A=QDQT.

    Def. Complex Numbers

    In cartesian form, z=a+bi

    In polar form, z=r(cosθ+isinθ)=reiθ

    The modulus is its magnitude |z|=a2+b2=r

    The principal argument, denoted arg(z), is θ if π<θπ

    The complex conjugate of z=a+ib is z¯=aib, and their product zz=|z|2 is a real number: a2+b2>0

    Thm z is a real number iff z=z

    Proof Let z=a+bi for a,bR. Then z¯=abi.

    () Assume that z is a real number. Then b=0z=a=z¯.

    () Assume that z=z. Then a+bi=abib=bb=0

    E.g. Find the modulus and principal argument for z=3i

    r=|z|=(3)2+(1)2=4=2

    arg(z)=θ=π/6

    E.g. z = -1 + i. Express z1 in the form a + bi where a and b are real numbers.

    11+i=11+i1i1i=1i1+1=1212i

    Def. De Moivre's Theorem

    (eiθ)n=einθ and (reiθ)n=rneinθ where n is a positive integer

    Def. nth roots of z

    E.g. Solve z4=2(3i1)

    RHS is 2+23ir=4+43=4,b=2,h=23θ=2π3

    RHS in polar form is 4ei(2π3)

    LHS in polar form is (reiθ)4

    Eq'n in polar form is (reiθ)4=4ei(2π3)r4eiθ4=4ei(2π3)

    So r4=4r=2 and 4θ=2π3+2kπθ=π6+π2k,kZ

    k = 0 z=2e(π/6)i=2(32+12i)

    k = 1 z=2e(2π/3)i=2(12+32i)

    k = 2 z=2e(7π/6)i=2(3212i)

    k = 4 z=2e(5π/3)i=2(1232i)

    Formula for nth roots of z=r(cosθ+isinθ):

    r1n(cos(θn+2kπn)+isin(θn+2kπn)) for k = 0, 1, 2, …, n-1

    E.g. Find the 4 fourth roots of 1.

    Method 1: Factor and solve.

    z41=0(z21)(z2+1)=0z=±1 or z=±i.

    Method 2: Use the formula. θ=0,r=1,n=4

    k=0cos0+isin0=1k=1cosπ2+isinπ2=ik=2cosπ+isinπ=1k=3cos3π2+isin3π2=i

    E.g. Find eigen values/eigen vectors of A=[1111]

    p(λ)=(λ1)2+1=λ22λ+2=0

    λ=2±482=1±i

    λ1=1+i

    AλI=[1(1+i)111(1+i)]=[i11i]

    {R1=iR1=[1,i]R2=R2i(R1)=[1+i2=0,i+i=0][1i00]a=ibv1=[i1]

    λ2=1i

    v2=v1=[i1]

    Def. Complex Vector Space

    Cn= space of ordered n-tuples of complex numbers. Vectors in Cn are [a1+b1ian+bni]. Assume scalars are in C

    Real scalarsComplex scalars
    dim(Cn)=2ndim(Cn)=n
    basis = {[100],...,[001],[i00],...,[00i]}basis = {[100],...,[001]}

    A complex vector space is one in which scalars are complex numbers.

    E.g. M2,2(C)={[abcd]:a,b,c,dC} is a complex vector space (with usual operations)

    E.g. R2 is not a complex vector space since it is not closed under scalar mult (vR2,ivR2)

    E.g. If v=[11], find vB if B=([ii],[i1]) is a basis of C2

    [11]=z1[ii]+z2[i1]

    [ii1i11][ii101i2][11i01i2][11i0121i1+i1+i=1+i][101011+i]

    {z1=1z2=1+ivb=[z1z2]=[11+i]

    Note {[ii],[i1]} is linearly independent since the matrix has rank 2.

    Def. Conjugate Transpose

    The conjugate transpose of A is defined as A=AT

    Properties

    Let A, B be m x n matrices and kC

    1. (A)=A
    2. (A+B)=A+B
    3. (kA)=kA
    4. (AB)=BA if A,B are square

    Def. Euclidean Inner Product / Dot Product

    In Rn:xy=x1y1+...+xnyn=yTx and x2=xx

    In Cn:x,y=x1y1+...+xnyn=yx

    E.g. Find u,v where u=[i01+i] and v=[3i72i]

    u,v=(i)(3i)+(1+i)(2i)=(i)(3+i)+(1+i)(2i)=3i12i+2=1+i

    Complex Dot Product Properties

    Let u,v,wCn,kC

    1. u,u0 and u,u=0 iff u=0
    2. u,v=v,unot commutative! order matters!
    3. u+v,w=u,w+v,w and w,u+v=w,u+w,v
    4. ku,v=ku,v and u,kv=ku,v

    Def. Euclidean Norm

    The Euclidean norm (or length) of uCn is ||u||=u,v

    Def. Unitary Matrix

    A complex matrix U is said to be unitary if any of the following is true (analogous to orthogonal matrices in Rn)

    Property the product of unitary matrices is unitary. AA=BB=IAB(AB)=ABBA=AIA=I

    Def. Hermitian Matrix

    H is a Hermitian matrix if H=H (analogous to symmetric matrices in Rn).

    • It has real eigen values and is diagonalizable (by a unitary matrix).
    • Eigen vectors corresponding to distinct eigen values are orthogonal.
    • A=AAz,w=z,Aw  w,z,Cn

    Proof (eigen values are real)

    Let λC be an eigen value and H be a Hermitian matrix.

    Then Hx=λx for some x0

    Since H is Hermitian, H=H

    LHS: xTHx=xTλx=λ(xTx)

    RHS: xTHx=(Hx)Tx=(λx)Tx=λ(xTx)

    Equating LHS and RHS, we have that λ=λλ is real

    Proof (eigen vectors are orthogonal)

    Let λ1,λ2 be eigen values of A with (nonzero) eigen vectors v1 and v2, then Av1=λ1v1,Av2=λv2

    Av1,v2=λ1v1,v2=λ1v1,v2 and Av1,v2=v1,Av2=v1,λ2v2=λ2v1,v2, so λ1=λ2

    (If we take v1=v2, λ1=λ2, then λ1=λ1,λ2=λ2 which also proves the eigs are real)

    Since the eigs are real, λ2=λ2λ1v1,v2=λ2v1,v2(λ1λ2)v1,v2=0

    So if λ1λ2, then v1,v2=0v1v2

    Proof (Hermitian iff Az,w=z,Aw)

    () If A=A, then Az,w=w(Az)=WAz=(Aw)z=z,Aw

    () Let A=[aij]. Then aij=eiTAej=eiAej=Aej,ei=ej,Aei=(Aei)ej=eiTAej=aji

    Note if A is skew-Hermitian(A=A), then iA is Hermitian, and vice versa.

    Fact Every square matrix can be decomposed into the sum of a Hermitian matrix and skew-Hermitian matrix

    C=12(C+C)Hermitian+12(CC)skewHermitian

    Def. Normal Matrix

    A square matrix A is normal if it commutes with its conjugate transpose: AA=AA

    Property

    Shortcuts (to prove a matrix is normal)

    Def. Unitarily Diagonalizable

    A square matrix A is unitarily diagonalizable iff A is a normal matrix.

    E.g. Determine all values of aC such that A=[ia2i] is unitarily diagonalizable.

    By thm, A must be normal, so AA=AA

    [ia2i][i2ai]=[i2ai][ia2i]

    Calculate top left entry on both sides i(i)+aa=i(i)+44=aa4=|a|2|a|=2

    Hence a is any number x+iy such that x2+y2=4

    E.g. Find all aC such that the matrix [i4ai] is unitarily diagonalizable.

    By thm, A must be normal, so AA=AA

    [i4ai][ia4i]=[ia4i][i4ai]

    1+16=1+a2|a|=4

    Hence a is any number of the form x+iy such that x2+y2=4

    W10 (9.3)

    Comparison - Real vs Complex

    Real (Symmetric matrix)Complex (Hermitian matrix)
    eigen values of A are realeigen values of H are real
    eigen vectors corresponding to distinct eigen values are orthogonal in Rneigen vectors corresponding to distinct eigen values are orthogonal in Cn
    there exists an orthogonal matrix Q s.t. QTAQ is diagonalthere exists a unitary matrix U s.t. UHU is diagonal

    Notes (eigenvalues)

    If a matrix has odd order (order = m x n), then the eigen values must be real.

    For a real matrix, eigen values come in conjugate pairs, i.e. if a+ib is an eigen value, then aib is also an eigen value.

    For an orthogonal matrix, the eigen value pairs all lie on the unit circle.

    For a Hermitian matrix, eigen values lie on the real axis; for a skew-Hermitian matrix, eigen values lie on the imaginary axis.

    Proof (eigenvalues of skew-Hermitian matrices have form ib,bR)

    Let v be an eigen vector for A with corresponding eigen value λ=a+ib

    Av,v=vAv=vλv=λvv

    Av,v=v(A)v=(Av)v=(λv)v=λvv

    So λ=λa+ib=(a+ib)=(aib)=a+iba=aa=0

    If we replace Av,v with Av,w, then we can prove that v,w=0 (eigen vectors corresp to dif eigen values are orthogonal)

    Def. Schur's Theorem

    If A is an n x n complex matrix, then there must exist a unitary matrix U s.t. U1AU=UAU=T is upper triangular. (The main diagonal of T are eigenvalues of A)

    Def. Spectral Theorem

    Recall Let A be an n x n real matrix. A is orthogonally diagonalizable iff A is symmetric.

    Let A be an n x n complex matrix. A is unitarily diagonalizable iff A is normal.

    Lemma If A is Hermitian, then there exists a unitary matrix U such that UAU is diagonal.

    Proof By Schur's thm, there exists an upper triangular matrix T=UAU.

    Then T=(UAU)=UA(U)=UAU=TT=T

    Since T is both upper triangular and lower triangular, it must be diagonal

    Note The converse is false.

    E.g. A=[0110] is not Hermitian but it is unitarily diagonalizable.

    W11 (9.4)

    Def. Jordan Canonical Form

    Every n x n complex matrix A is similar to a matrix of the following form (λi's are not necessarily distinct).

    The matrix is composed of Jordan blocks along the diagonal.

    Def. Block Matrix

    A block matrix is one that has been partitioned into blocks.

    E.g. Find the eigen values of A=[011000230004]

    Observe that A can be partitioned into 2 x 2 blocks.

    The top left block has eigs ±i, and the bottom right block has 2 and 4.

    Properties of Jordan Blocks

    Let J be a k x k Jordan block [λ10...00λ1λ010......0λ]

    Fact

    (JλI)ei=ei1 for 1<ik

    (JλI)e1=0e1null(JλI) note that e1 is an eigen vector with rank 1

    (JλI)e2=e1(JλI)2e2=(JλI)e1=0e1,e2null(JλI)2

    (JλI)pep=0e1,e2,...,epnull(Jλ)p

    Def. Generalized Eigenvector

    For an n x n complex matrix A, a generalized eigenvector with rank p is a non-zero vector xCn corresponding to eigenvalue λ if (AλI)px=0 for some positive integer p.

    Generalized eigenspace: set of all generalized eigenvectors for λ plus 0. E^λ={xCn:(AλI)px=0 for some p}

    Notes

    Why find generalized eigenvectors?

    By forming P using generalized eigenvectors, we can turn A into JCF using P1AP

    E.g. Find generalized eigenvectors for A=[110012003]

    Eigen values are 1, 1, 3

    λ=3E3=sp([122])

    λ=1E1=null([010012003])=sp(e1)

    When p=1, (AλI)x=0x=e1

    When p=2, (AλI)2x=0x=null((AλI)2)=null([010002003]2)=null([001000000])

    ={[st0]:s,tC}=sp(e1,e2) where e2 is a generalized eigen vector

    Thm (Similar to JCF)

    Every square complex matrix is similar to a JCF.

    Properties

    E.g. If a 7 x 7 matrix A has eigenvalues {1, 1, 1, 2, 2, 2, 2}, what are the possible JCF's?

    For λ=1, the possible configurations are:

    For λ=2, the possible configurations are:

    For small order, the geo mult gives the block structure, but it does not tell us the structure for alg mult = 4, geo mult = 2 (see above underbrace). How do we tell which structure to use?

    Compute dim(null(AλI)k) for k1, i.e. generalized eigenspaces.

    Note Each k x k Jordan block has

    In general, an arbitrary matrix has a chain for each block in its JCF.

    E.g. For a 5x5 matrix, an eigen value with alg mult = 5, geo mult = 3, we need 2 generalized eigenvectors:

    E.g.

    dim(null(B))=9 for the last one

    E.g.

    A=(λ100λ100λ00λ10λ)

    For the 1st block, we have the chain v1=[10000]v2=[01000]v3=[00100]

    For the 2nd block, we have the chain w1=[00010]w2=[00001]

    v1,w1null(AλI) are independent eigenvectors or rank 1

    v2,w2null((AλI)2),null(AλI) are independent generalized eigenvectors of rank 2

    v3null((AλI)3),null((AλI)2) is an independent generalized eigenvector of rank 3

    Put them into a matrix P=[v1|v2|v3|w1|w2] and J=P1AP

    Fact dim(null(AλI)k)= # ind. gen. eigenvectors of rank at most k

    dim(null(AλI))=2 since it has v1,w1

    dim(null(AλI)2)=4 since it has v1,v2,w1,w2

    dim(null(AλI)3)=5 since it has v1,v2,v3,w1,w2

    dim(null(AλI)k)=5 k3

    Fact dim(null(AλI)k)dim(null(AλI)k1) = # ind. gen. eigenvectors of rank k = # Jordan blocks of size k

    # of Jordan blocks of size k = (# Jordan blocks of size k) - (# Jordan blocks of size k+1)

    =dim(null((Aλk)k))dim(null((Aλk)k1))(dim(null((Aλk)k+1))dim(null((Aλk)k)))

    =2dim(null(AλI)k)dim(null(AλI)k1)dim(null(AλI)k+1)

    E.g. Find Jordan Form of A if p(λ)=(λ3)13 and suppose:

    dim(null((A3I)1))=6 so # ind. gen. eigenvectors of rank at most 1 = 6

    dim(null((A3I)2))=10

    dim(null((A3I)3))=12

    dim(null((A3I)4))=13

    So A is 13 x 13, λ=3 with alg mult 13

    Obtain 6 chains, each corresponding to a Jordan block (each eigen vector produces a Jordan block), and the length of the chain = size of the block

    Use formula: 2dim(null(AλI)k)dim(null(AλI)k1)dim(null(AλI)k+1)

    dim(null((A3I)1))=6 2(6) - 0 - 10 = 2 blocks of size 1

    dim(null((A3I)2))=10 2(10) - 6 - 12 = 2 blocks of size 2

    dim(null((A3I)3))=12 2(12) - 10 - 13 = 1 block of size 3

    dim(null((A3I)4))=13 2(13) - 12 - 13 = 1 block of size 4

    dim(null((A3I)5))=13 2(13) - 13 - 13 = 0 block of size 5

    E.g. Is it possible for a matrix to have p(λ)=(λ7)5 and

    dim(null((AλI)2))=2

    dim(null((AλI)3))=3

    dim(null((AλI)4))=5

    Def. Cayley Hamilton Thm

    Every square matrix satisfies its own characteristic polynomial.

    E.g. Let A=[1234] where p(λ)=λ25λ2=0. Find A1 (in terms of powers of A).

    Then by Cayley Hamilton, p(A)=A25A2I=[0000]

    2I=A25A2A1=A5IA1=12(A5I)=12[4231]

    E.g. Let A=[2152]. Find A6.